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Abstract 

We present results of a search for D° — D° mixing and doubly-Cabibbo- 
suppressed decays of the D° in Fermilab experiment E791, a fixed-target 
charm hadroproduction experiment. We look for evidence of mixing in the 
decay chain D* — > irD — > it (Kit or Kit-kit). If the charge of the pion 
from the D* decay is the same as the charge of the kaon from the D decay 
(a "wrong-sign" event), mixing may have occurred. Mixing can be distin- 
guished from other sources of wrong-sign events (such as doubly-Cabibbo- 
suppressed decays) by analyzing the distribution of decay times. We see no 
evidence of mixing. Allowing for CP violation in the interference between DCS 
and mixing amplitudes our fitted ratio for mixed to unmixed decay rates is 
r mix = (0.39^:1 ± 0.16)%. This corresponds to a 90% CL upper limit of 
r-mix < 0.85%. The sensitivity of this result is comparable to that of previous 
measurements, but the assumptions made in fitting the data are notably more 
general. We present results from many fits to our data under various assump- 
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tions. If we assume T m i x — 0, we find a two-sigma wrong-sign enhancement 

in the Ktt mode which we ascribe to doubly Cabibbo-suppressed decays. The 

ratios of doubly Cabibbo-suppressed decays to Cabibbo-favored decays are 

rdca (Kir) = (0.681^3 ± 0.07)% and r dcs {KTnnr) = (0.25±g;|t ± 0.03)%. 
12.15.Ff, 13.25.Ft, 14.40.Lb 
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I. INTRODUCTION 



The Standard Model predicts a rate for D° — D° mixing which is many orders of magni- 
tude below the reach of present experiments. Typical calculations Q give r mix , the ratio of 
mixed to unmixed decay rates, in the range 1(T 10 to 10" 7 . In contrast, various extensions to 
the Standard Model allow a mixing rate close to the current experimental sensitivity of 
10 -3 to 10 -2 . Consequently, a discovery of D° — D° mixing at currently measurable levels 
would be inconsistent with the Standard Model, and would provide a clear signal for new 
physics. 

Experimentally, mixing is identified by a change in the charm quantum number of the 
neutral D meson between its production and decay. In the analysis presented in this paper, 
the charm of the produced D is determined from the decay D* + — ► D°tt + (or — > D °7r _ ), 
where the charge of the pion indicates whether a D° or a D° was produced. D decays 
are reconstructed in four all-charged hadronic decay modes D — > K~tt + , D — > K + tt~, 
D — > K~7t~7t + 7t + or D — > K + 7t + 7t~7t~ . (Hereafter, we will omit the charge superscripts 
from the final states where context allows.) Possible evidence for mixing comes from the 
detection of a meson produced as a D° (D °) decaying to a "wrong-sign" final state which 
contains a K + (K~), with the kaon charge opposite to that expected for unmixed decays. 

Fermilab experiment E691 |5| has previously used this technique to set what is cur- 
rently the strictest upper limit on mixing, r mix < 0.37%, albeit with specific assump- 
tions which we will address in this paper. Fermilab experiment E615 obtained a limit 
of r m i X < 0.56% by looking for same-sign muon pairs in 7r-tungsten interactions, based on a 
specific model for charm production jjj. Evidence for wrong-sign decays has been presented 
by the CLEO collaboration |J, which measures the ratio of wrong-sign to right-sign decays 
to be (0.77 ± 0.25 ± 0.25)% for the Kir final state. However, the CLEO experiment was 
unable to distinguish between mixing and doubly- Cabibbo-suppressed decays, which also 
produce wrong-sign events. Recently we have reported on a search for mixing using semilep- 
tonic decays of the D° (D°) which do not have a doubly-Cabibbo-suppressed background 
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§. We found that r mix < 0.50%. 

It is possible to distinguish doubly-Cabibbo-suppressed (DCS) and mixing contributions 
to the wrong-sign rate by studying the distribution of D decay times. In the limit of small 
mixing, the rate for wrong-sign decays takes the form 



T[D°(t)^f] = ^\(f\H\D°) CF \ 2 



2 

Q 



V 

4|A| 2 + ( (AM) 2 + J t 2 + (2i?e(A)Ar + AIm(X)AM)t\ , (1) 



where 



p (f\H\D°) DCS 

q(f\H\D% F ' {) 
and p and q describe the relationship between the charm eigenstates \D°) and \D °) and the 
mass eigenstates |-Di j2 ): 

= p\D°) + q\D°), 
\D 2 )=p\D°)-q\D°). (3) 

The amplitude (f\H\D°) ncs represents the DCS decay of the D° while (f\H\D°)cF is the 
Cabibbo-favored counterpart for the decay of D °. The parameters AM and Ar describe 
the differences in mass and width of the two physical states. The term proportional to |A| 2 
in Eq. (H) describes the contribution from DCS amplitudes, the term proportional to t 2 de- 
scribes the lowest-order contribution from mixing, and the term proportional to t represents 
the interference between mixing and DCS amplitudes. We can apply this formula to the 
measured time distribution of wrong-sign decays to determine the separate contributions 
from DCS and mixing amplitudes. 

In the study that follows, we examine a sample of about 9,100 reconstructed, tagged 
D° decays to look for wrong-sign decays, using the different time distributions to separate 
the DCS and mixing contributions in our search. As we shall see, there are no significant 
wrong-sign signals in our data, which leads us to set restrictions on the ratio of wrong- 
sign to right-sign rates. The most likely fit (in the possible Standard Model scenarios) 



will be presented first. Afterwards, we will determine the effects of relaxing all constraints 
and of additional constraints (absence of DCS-mixing interference, no mixing at all) which 
investigate interesting physics cases or are necessary to compare with previously published 
results. 



II. EFFECTS OF CP VIOLATION 

Equation ([!]) describes the rate for D° to decay to a wrong-sign final state /. Within 
the context of some new physics models, it is possible that the rate for D to decay to / 
is not the same, and that CP is violated to a significant extent. Thus, it is important to 
allow for the possibility of CP violation. This results in the most conservative upper limit 
on wrong-sign decays. The analysis presented here is the first experimental study to allow 
for the possibility of CP violation. (For recent discussions of the role of CP violation in 
D°-D° mixing see §, §.) 

Formally, the conjugate equation is 



T[D°(t)^f} = e -^\(f\H\D°} CF \ 



2 
p 



x 4 1 A | 2 + (jAM) 2 + ^£ j t 2 + (2i?e(A)Ar + AIm(X)AM)t] , (4) 



with 



q (f\H\D°) DCS 

p (f\H\D°) CF ' 1 ' 

In principle, CP violation can arise through a difference between Equations ([!]) and (£|) in 
any one of the three terms. Any term in (|1|) can differ from its charge conjugate in (|J) 
as a result of the interference of two or more contributing amplitudes which have non-zero 
relative phases of both the CP-conserving and CP- violating type. 

2 2 

Inequality of the two constant terms (i.e., 2 | A | 2 ^ 2 |A| 2 , but see comment 0) is 
referred to as direct CP violation. This could be significant if two or more comparable DCS 
amplitudes contribute with different CP-conserving and CP-violating phases. However, the 



Standard Model contribution (which is expected to dominate) provides only one weak, CP- 
violating phase. Direct CP violation is therefore likely to be small. Similarly, the two charge 
conjugate terms proportional to t 2 will be the same unless there are two or more mixing 
amplitudes with relative CP-violating and CP-conserving phases. On the contrary, most 
models suggest that if mixing occurs at all, it is likely to be dominated by a single CP- 
violating phase. Therefore, the most plausible constraint involving CP violation restricts 
CP violation to the interference term. We will explore this possibility, as well as the more 
general case without CP restrictions, in the study of our data which follows. 

III. DESCRIPTION OF E791 

We report the results of a search for D° — D mixing and DCS decays using hadronic 
decays found in data from our experiment, Fermilab E791. We collected approximately 
2 x 10 10 hadronic interactions in the 1991-2 fixed-target run using the TPL spectrometer 
|T0|j with a 500 GeV/c tt~ beam. There were five foil targets: one 0.5-mm thick Pt foil 
followed by four 1.6-mm thick diamond foils with 15 mm center-to-center separations. This 
arrangement allowed us to greatly reduce secondary interaction backgrounds by selecting 
only charm candidates which decayed in air. 

The target region was preceded by six planes of silicon microstrip detectors and 8 pro- 
portional wire chambers (PWC's) used for beam tracking and was followed by 17 additional 
planes of silicon microstrip detectors for measuring tracks produced at and downstream of 
the primary vertex. The track momenta and slopes were also measured in the downstream 
spectrometer which had two magnets, 35 planes of drift chambers, and two PWC's. Two 
threshold Cerenkov counters provided tt/K separation in the 6-60 GeV/c momentum range 

[n|. 

The mixing analysis in this paper relies heavily on track reconstruction, which begins by 
using hits in the silicon detector and folds in additional information from the downstream 
devices. The tracking efficiency is approximately 80% for particles with momenta greater 
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than 30 GeV/c and drops to around 60% for particle momenta of 10 GeV/c. The mean 
number of reconstructed tracks used to fit the primary vertex is seven. After reconstruc- 
tion, events with evidence of multiple vertices were kept for further analysis. The list of 
reconstructed vertices is used in the selection criteria described below. 

We determined our production (primary) and decay vertex resolutions by comparing 
reconstructed and true vertex positions using our Monte Carlo detector simulation. The 
transverse resolutions quoted below are one-dimensional values. Longitudinal and transverse 
position resolutions for the primary vertex are 350 and 6 /im, respectively. For the mean 
D° momentum of 65 GeV/c the longitudinal resolutions for Kir and Knmr vertices are 320 
and 395 /im, respectively, and increase by 33 and 36 /im, respectively, for every 10 GeV/c 
D° momentum. Similarly, for the mean momentum, 65 GeV/c, of the observed D 0, s, the 
transverse resolutions for Kn and Kunn vertices are 10 and 12 /im respectively and decrease 
by about 0.5 /im for every 10 GeV/c increase in D° momentum. 

The kaon and pion identification efficiencies and misidentification probabilities vary with 
momentum and with the signatures we require in the Cerenkov detector. For typical particle 
momenta in the range 20 GeV/c to 40 GeV/c, the Cerenkov identification efficiency of a 
kaon is around 58% when the probability for a pion to be misidentified as a kaon is 4%. In 
the same momentum range the Cerenkov identification efficiency of a pion is around 93% 
when the probability for a kaon to be misidentified as a pion is 35%. 

IV. SELECTION OF DATA SAMPLE 

A search for the rare wrong-sign mixing and DCS decays requires selection criteria that 
emphasize background reduction. We achieve this goal in two stages: initially reconstructing 
displaced secondary vertices and using a few loose criteria for selecting D° decays to reduce 
the data sample, and then optimizing the data selection with artificial neural networks. 

Initial reduction of the large E791 data set to a manageable size was achieved with the 
aid of a few simple criteria. Here, we describe the cuts made in these initial stages for 
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the D° — > Kn mode. (When we refer to Kn or Knnn in this paper, without any explicit 
signs, we include the charge conjugate states. Otherwise, we indicate a specific final state by 
explicitly specifying at least the kaon charge or specifying right-sign (RS) or wrong-sign (WS) 
decays.) Two-prong vertices were used to start the search for D° decays. The invariant mass 
of the two-prong D° candidate, assumed to be Kn, was required to be in the range 1.7 to 2.0 
GeV/c 2 . The kaon candidate was chosen as the one with the higher probability of being a 
kaon based on Cerenkov information. To further reduce the contributions from misidentified 
D° decays, Kn candidates were rejected if the reverse hypothesis (nK) fell within 2 a of 
the D° mass, where a is the measurement resolution for the D° mass. Similarly, to reduce 
contamination from D° decays to K + K~ and n + n~ , Kn candidates were rejected if the 
K + K~ or n + n~ mass hypotheses fell within 2 a of the D° mass. To help ensure that the 
reconstructed secondary vertex was a true decay vertex, we required that it be separated 
from the primary vertex by at least 8 standard deviations (<ta 2 ) in the beam direction (i.e., 
Az/ctaz > 8). Two further requirements ensured that the reconstructed D° was consistent 
with originating at the primary vertex. First, the impact parameter of the reconstructed 
D momentum with respect to the primary vertex, b p , was required to be less than 60 /xm. 
Second, the component of the reconstructed D momentum perpendicular to the D line-of- 
flight (as determined from the primary and secondary vertex positions), p® , was required to 
be less than 0.35 GeV/c. The K and n decay tracks were required to be well-reconstructed 
in the silicon detectors and drift chambers. Finally, the momentum asymmetry of the K 
and n as measured in the laboratory frame (i.e., p asy = \px — p n \/\pK + Pn\) was required 
to be less than 0.65. This reduced the contribution from random track combinations, which 
tended to be asymmetric. 

The cuts for the D° — > Knnn case were similar in the initial stages. We used candidates 
arising from both 4-prong vertices and 3-prong vertices (with an added track). The two 
vertex samples contributed roughly equal amounts to the signal. We required Az/cr^z > 8, 
b p < 60 /mi, p® < 0.5 GeV/c, and that the decay vertex was outside the target foils. The 
D° candidate mass was required to be in the range 1.7 - 2.0 GeV/c 2 . To eliminate reflections 
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from Cabibbo-favored decays we examined the hypothesis that the kaon was actually a pion 
and one of the pions opposite in charge to the kaon was actually a kaon. Since there are 
two such possibilities, candidate Kititit decays were rejected if either possibility yielded a 
candidate mass within 2a of the D° mass. Tracks were required to have momenta greater 
than 0.5 GeV/c and to have greater than 4cr of transverse separation from the primary 
vertex where a is the measurement resolution on the separation. Finally, the decay tracks 
were required to form a vertex that was no more than 2.5 cm downstream of the final target. 
Beyond that point the silicon detectors and other material in the beam path provided large 
numbers of secondary interactions. 

In the final stage of analysis, we used two-layer feed-forward neural networks to optimize 
the signal selection |T2|JT3[| . Specifically, we chose selection criteria that maximize S/yfB 
where S and B were the signal and the background under the signal for the right-sign 
decays. A vector, whose components are variables such as the ones just mentioned, was fed 
into each neural net as the input layer. Each node in the next (hidden) layer computed 
the sigmoid of the sum of an offset and the inner product of the input vector with a weight 
vector. The results from this layer in turn formed the input for a single node in the final 
(output) layer. Thus, the networks effectively combined information from each variable we 
would otherwise have "cut" on and provided a single output value in the range to 1. This 
output was monotonically related to the probability that a given candidate was signal and 
not background. 

Since our two major sources of background were false D° candidates and real D° candi- 
dates combined with random pions to produce fake D* candidates, we used separate neural 
networks to classify the D° and D* candidates. Although there are only two modes of D° 
decay examined in this work, three D° samples were used to train the neural nets: one for 
the D° — > Kit mode and two for the D° — > Kit-nit mode. The two separate D° — > Kit-Kit 
samples contained candidates from vertices that had either all four or only three of the four 
tracks. 

In order to minimize our dependence on Monte Carlo, we used D° candidates in our real 
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data to train separate neural nets for each of the three samples. We chose D° candidates that 
do not combine with pions to give a D* candidate. The training sample is thus independent 
of our mixing sample and ensures that the neural net training was unbiased. A fourth neural 
net was trained using part of the right-sign D* + — > D°n + sample to classify D* + candidates. 
Every net was trained using events in the peak region as "signal" and the remaining events 
as "background". We selected only those events for which the product of the D° and D* 
net outputs was greater than a certain value rather than making individual "cuts" on many 
variables. 

In the D° — > K~ir + mode, the net was presented with twelve input variables: the pr of 
the D° relative to the incident pion beam direction, the separation between the secondary 
and primary vertices (Az), p®, b p , the x 2 P er degree of freedom for the secondary vertex 
fit, the Cerenkov -based probability for the kaon to be a kaon, the momentum asymmetry 
(Pasy), the consistency probability for the secondary vertex to be in a target foil, the track fit 
X 2 per degree of freedom for the two tracks and the number of tracking systems traversed by 
each of the two tracks. The two nets for the D° — > K~7i + 7i + 'K~ mode used seven variables, 
of which the first six variables were the same as the first six just listed for the D° — > K~n + 
mode. The seventh variable was the smallest contribution to the x 2 of the fit to the primary 
vertex from any of the four decay tracks. Finally, the D* neural net was constructed using 
only five variables: the x 2 P er degree of freedom for the track fit for the pion from the 
D* + decay (referred to hereafter as the "bachelor pion"), the number of tracking stations in 
which the pion is detected, the probability that it is not a fictitious track, its momentum 
and the x 2 P er degree of freedom for the vertex fit of the D° and the bachelor pion. There 
were three nodes in the hidden layer for the D° — > K~tt + and D* nets and four nodes in the 
hidden layers of each of the D° — > K^n + n + n^ nets. 

Although we considered many variables, we pruned the list down to the variables listed 
above and also pruned some of the connections to the hidden layer nodes when their contri- 
butions to the output were deemed unimportant using a technique called subset reduction, 
implemented as follows. Nodes in a given layer were viewed as a linear array, one row for 
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each event. The matrix thus formed was subjected to singular value decomposition using 



QRcp factorization [TJJ] . The "energy content" of the nodes was determined by the resulting 
eigenvalues |15j. Nodes with an "energy content" < 1% were deleted. 

We also tried other techniques for selecting events, including the more common method 
of using independent "cuts" in each variable and a Binary Decision Tree (BDT) technique 
| 16fl . The sensitivity of the neural net technique was about 10% higher than the BDT in the 



D° — > Kir mode and about 30% higher than the BDT in the D° — > Kittttt mode; in turn 
the BDT was better than the commonly used "cuts" technique. One further advantage of 
the neural net technique was that the output could be used to choose the best candidate in 
an event, should there be more than one (a rare occurence). This simplified the statistical 
analysis in the fits to our data. 

The results of our neural net optimizations are shown in Figure [l| for right-sign and wrong- 
sign Kir and Kiririr final states. In this figure, we plot the candidate D° mass (m(KTr) or 
m(A"7T7T7r)) versus Q, defined as Q = mlKmr) — m(Kir) — m(7r) or Q = miKn-n-n-n) — 
m(K7i7i7i) — m(7r). For real D* decays, Q has a value of about 5.8 MeV. In the right-sign 
plots (top of Figure [I]), clear signals are apparent over small backgrounds. The bands of 
events at m(KiT) or m(K7nnr) « 1.865 GeV/c 2 are due to real D° decays combining with 
random pions in the event to give a false D* candidate. These bands are more readily seen in 
the wrong-sign plots (bottom of Figure [3]) where the vertical scale is expanded by a factor of 
20. This background, which we will refer to as "random pion" background, is the dominant 
one in our analysis. We will call the remaining broad background visible in the plots the 
"false D°" background. 

In the right-sign plot for the Kn mode, there is a signal of 5643 events above a background 
of 235 events. In the right-sign plot for the Kirmr mode, there is a signal of 3469 events 
above a background of 146 events. The signals and backgrounds are estimated in a region 
spanning ±1.75cr around the peak in Q, for 1.77 < m^o < 1.97 GeV/c 2 . The precise region 
used to estimate S and B is not important for the optimization. The resulting sensitivities 
(S/y/B) for the two modes are 368 (Kir) and 287 (Ktvtttt). 
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V. FIT TECHNIQUE 



The process just described results in eight separate datasets: D° or D °, decaying to Kit 
or Ktttttt, right-sign or wrong-sign decays. Although it is possible for us to fit each data set 
separately (which we have done as a check), it is useful to combine all eight data sets into 
a single fit. This allows us to take advantage of the fact that the central values of the D 
and D* signals, as well as the rriKn, mKmrn and Q resolutions, are the same for the different 
data samples. Under these circumstances, most of the parameters of the single fit (which 
are largely parameters to describe background) remain uncoupled, and in that sense are no 
different from eight separate fits. Only the signal masses and resolutions are constrained 
across data samples. Studies of separate fits for the different samples show no significant 
shifts from the single fit results and have convinced us that these constraints are valid. 

Our most general fit includes no constraints beyond those just described, and is summa- 
rized in Section |VII| and in Table [V|. However, as discussed in Section |I| the most likely 
scenario is that there is no CP violation in either the DCS or the pure mixing terms of the 
wrong-sign rates. This leads to three additional constraints, discussed at the end of this 
section, which then lead to the results of Table J|. These results are the main focus of our 
studies in mixing. Finally, in Section |V11| we perform other fits using additional physical re- 
strictions (no DCS-mixing interference, or no mixing at all) in order to explore other physics 
hypotheses and to compare with previous measurements. In what follows, we describe the 
terms of the fit in detail. 

As stated, we perform a single unbinned maximum likelihood fit to the data, using the 
following form for the In of the likelihood: 



In C = £ In A - £ ( i In [27rJV£j + ) 

i f V lly pred J 



(6) 



where the first sum is over all the D* candidates, the second sum is over the eight decays 
used in the analysis, Li represents the likelihood for each candidate, and N* bs is the observed 
number of candidates for each final state. The argument of the second sum is the logarithm 
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of a normalized Gaussian, and serves to constrain the number of candidates predicted by 
the fit, Np red . There are three contributions to each Cf. signal, random pions with real D 0, s, 
and random pions with false D 0, s. In addition, for the D° — > K^n^ samples we include a 
contribution from misidentified D° — > K + K~ and D° — > 7i + tt~ decays, which also contribute 
measurable background. 

d = S(mi,Qi,ti) + M(mi,Qi,U) + P{m u Qi,ti) + F{m u Q h U), (7) 

where mi, Qi, and U are the D mass, Q value and proper decay time of each candidate. A 
wrong-sign signal event is described by simple Gaussian terms in and Qi, multiplied by 
a sum of the three different decay time distributions that represent the DCS, mixing and 
interference contributions (see Eq. (|TJ)): 

S{mi, Q h U) = — ^ e -Ko- m! ) 2 /24 x _^ e -(QD*-Q> f/2*l x 

^^dcs B eX p{ti) -\- A m i x B m i x {ti) -\- A-int -Bjnt(^i)j') (8) 

where 

B exp {U) = B° exp e(U) J dt e-^ 2 l^l e ~ ri , 

B mtx (U) = B° mix e(U) J dt e-^' 2 ^ t 2 e~ Tt , (9) 
B mt (U) = Bl t e{U) J dt e-^l^l t e~ rt , 

e(ti) is the reconstruction efficiency and <j is the decay time resolution. Each B(ti) is 
normalized to unit integral so that A dcs , A mix , and A int can be interpreted as the num- 
ber of observed candidates of each type. The Gaussian smearing integrals are performed 
analytically with a smearing width do = 0.05 ps. 

The reconstruction efficiency e(t) is the first of three functions that must be modeled for 
the fit. It is desirable to measure this function using real data rather than using Monte Carlo 
simulation. Fortunately, this can be accomplished with a sample of right-sign events. Since 
there is no mixing contribution to the right-sign decay rate, the true decay time distribution 
for right-sign decays is proportional to e~ Tt with V = (0.415 ps) -1 |L7j]. The reconstructed 
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distribution is proportional to e(t) J dt e'^ 1 ^ 2 l 2a \ e~ rt . Therefore, dividing the measured 
right-sign distribution (corrected for non-D° background using sideband subtraction) by the 
known smeared exponential gives a distribution proportional to the efficiency [18| . Figure |2| 



shows the results of that measurement for both the Kir and Kkk-k final states, which we 
will use to represent e(t) in the fit. 

Despite the explicit mass cuts designed to reduce backgrounds from D° — > KK and 



D° — > mi decays described in Section [IV], some contamination remains. The misidentified 
D° — > KK and D° — > tttt events are described by 

M(m 4 , Qi, t t ) = — *— A KKtmr U{mi) V{Qi) B exp {t t ). (10) 

where the functions U(rrii) and V(Qi) are obtained from Monte Carlo simulations of KK 
and tttt reflections remaining after all cuts, including explicit mass cuts designed to minimize 
these reflections. The parameters A^K^n describe the number of events in the wrong-sign 
K + tt~ and K~tt + samples. Similar backgrounds for the Kirnn mode are not significant (see 
Section |V111| for further discussion). 



The random pion background is described by 

P{rTH,Qi,ti) = A, Rm Bexp{fA (n) 

pved \J Lit Or, 



The background shape in Q, represented by R(Q), is independent of the candidate D mass. 
We model this shape by combining a D° candidate from one event with a 7r from another 
event. As long as the D° is not strongly correlated with other tracks in the event, and the 
selection cuts are not dependent on the spatial relation between the D° and other tracks 
in the event, this technique should provide an accurate model of background. Monte Carlo 
studies confirm the validity of this method. The resulting distribution is compared to the 
wrong-sign D° — ► Kit and D° — > Knim data samples in Figure |^. 

The false D° background is adequately described by a linear function in mf 

Qi, U) = -J- A ° (1 + A ! K ~ mo)) RiQi) B false (U), (12) 
N pred Am 
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where mo is an arbitrary reference chosen to be 1.87 GeV/c 2 and Am is the D° mass interval 
(0.2 GeV/c 2 ). The function R(Q) is observed to be the same as in the case of the random 
pion and real D° background described by Equation ( |TH ) above. The function Bf a i se (ti) 
describes the time distribution of the false D° background. We model this distribution 
using candidates from the D mass sidebands of the right-sign event sample (Figure |]). 
Since this background is very small, we do not need to model it with great precision, and 
the statistics of Figure £| are adequate. 

The likelihood function for right-sign decays is constructed similarly. Since right-sign 
decays were used to model e(t) and Bf a i se (t), we do not use the lifetime information for 
these events in the fit. Moreover, right-sign decays are not subject to mixing or interference, 
so the fit functions for these events are given by the simplified formulae 

S(m h Qi) = ^_ e -(m D o- m ^/2al x _^_ e - { Q D ,- Ql? /2al ^ ^ 



Npred \/27r (X £> V27T (X q 

P(mi, Qi) = -jj— A„ _^ e -(™ D «-^) 2 /^l (13) 

Npred V 2,71 <7j) 

F (»„«,) = ^- ■ 4o(1 + j4 i (m -""° )) m)- 

N pred Am 

We fit all the data, both right-sign and wrong-sign D° — > Kit and D° — > Ktttttt, simul- 
taneously in one fit. Separate terms for charge conjugate final states are provided to allow 
for the most general possible form for CP violation. Under these conditions, we have four 
signal parameters (A dcs , A mix , A int and A KKt7rn ) and three background parameters (A w , A 
and Ai) for the two wrong-sign decay modes D° — > K + tt~ and D° — > K~tt + . We have three 
signal parameters (Ad cs , A mix and A int ) and three background parameters (A n , A and A\) 
for the two wrong-sign decay modes D° — * K + -k-k'k and D° —>■ K^tttttt. For each right-sign 
mode (D° ^ K + 7T~, D° —>■ K+tttttt, D° — > K~tt + , and D° — > K^tttttt) we have one signal 
parameter (A rs ) and three background parameters (A w , A and Ai). Additionally, we have 
five mass parameters (m D o, <tktt, &Knwn, Qd*, &q) to describe the signal Gaussian functions. 
Separate D° mass resolutions are used for the Kir and Kinnt final states. The resolution in 
Q is dominated by the bachelor pion, and is therefore the same for the two final states. 
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With this list we have 47 parameters for a complete description of the data. However, we 
expect that the false D° backgrounds for right-sign and wrong-sign decays to the same D° 
final state should have the same slope parameter (Ax), although the level (Aq) may differ 
since they are combined with pions of different charge to form the D* candidates. This 
observation reduces the number of parameters to 43. 

We also note that the values r mix (D and r mix (D — > D°) should be independent 

of the D decay final state. Thus, without loss of generality, we can assume that r m i X (D° — > 
K+TT-) = r mix (D° -> K + ixnit) and r mix (D° -> R-H+) = r mix (D° -> fT™). This 
eliminates two more parameters from our fit |19|, leaving us with a total of 41 independent 



parameters to describe the full data set. 

It is convenient to express the wrong-sign signal parameters Ad cs , A m i X and Ai nt in terms 
of the ratios of produced wrong-sign events to produced right-sign events, since these are 
the parameters of primary physics interest. For the wrong-sign K~7t + final state 

r dcs (K~n + ) 



Adcs{D - 


-» K~7l + ) 


A rs (D° - 


> K + TT-) ' 


A • (T)° 

-fornix y-Ls 


-> K-iT + ) 


A rs (D° - 


-»■ K+TT-) 


A int (D - 


-> K-iT + ) 


As(D° - 


> K + TT-) 



r mix (K~^) = f^r^J^ x c mix , (14) 
+ , _A int (D ^ K-7T+) 

Tmt[K 71 > ~ A rs (D° - k+*-) x 

and similarly for fT + 7r~, i^~7r7T7r and i^ + 7T7T7r. The c's in the expressions for r mix and r int 
are given by 

/ dti Te{ti) J dt e- rt e -('-'*) 2 /2- 2 



Cint 



J dU |r 3 e(ti) J dtt 2 e~ rt e -(*-**) 2 /2^o 2 



(15) 



/ T 2 e(ti) / dt t e~ rt e -(*-*i) a /2«S ' 
These terms correct for the different integrated efficiencies for reconstructing wrong-sign 
DCS, mixing and interference events. Table [I] shows these correction factors for both the 
Kir and Ktttctc final states. 

Although the production characteristics of D° and D° are different in our experiment, 
the ratios in Equation ( fl4l) are designed to cancel this effect. In constructing these ratios we 
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implicitly assume that the Cabibbo-favored amplitudes (f\H\D°) and (f\H\D°) are equal 
in magnitude, as mentioned previously 0. With this assumption, r mix of Equation (|14D can 
be interpreted according to convention as 



1 



2P 



((AM) 2 + ^j. (16) 



At this point, the fit is completely general, with no physics assumptions applied. 
However, as discussed in Section [T| it is unlikely that CP is violated in the DCS 
and pure mixing terms of the wrong-sign rates, even in most extensions to the stan- 
dard model. Under these circumstances, there are three additional constraints, namely 
r dcs (D° -> K+n-) = r dcs (D° -> K~>k+), r dcs (D° -> K + m ) = r dcs (D° -> K~ m ), and 
r m i X (D° — > D°) = r mix (D° — > D°). These constraints remove three more parameters from 
the fit, leaving a total of 38. We will use this fit to give us our primary result, summarized 
in Table 0, 



VI. RESULTS 

We fit the data over the range 1.77 to 1.97 GeV/c 2 in m D , 0.0 to 0.020 GeV/c 2 in Q 
and 0.0 to 4.0 ps in t. Tables [IT] and |TTT| show the resulting 38 parameters from our primary 
fit, described in the previous section. The wrong-sign ratios are all small or consistent with 
zero, indicative of small DCS to Cabibbo-favored ratios and very little mixing. Using the 
criterion Aln£ = 0.82 (neglecting systematic errors), we calculate the one-sided, 90% C.L. 
upper limit for mixing to be r mix < 0.85%. There is also no evidence for CP violation. 
Figures |^ and ^| show the fit results overlaid on the data distributions for mo, t, and Q. 
Good agreement is evident in every distribution. 

The lego plots of Figure [I] demonstrate that the largest background comes from real D 
decays combining with random pions to produce false D* candidates. This phenomenon is 
also apparent in Figures |5| and ^| where we see many wrong-sign candidates accumulated at 
the D° mass (left column) but very few of these candidates show the correct Q value (right 
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column) to have come from D* decays. A true wrong-sign signal from mixing or DCS decays 
would be manifest as a simultaneous peak in both the M^n (or Mjcttttw) and Q distributions. 

It is important to note that the excess of candidates at Q ~ 0.006 GeV in the wrong- 
sign decays D° — > K + ir~ and D° —>■ K~n + (lower right plots of Figure |5|) is due primarily 
to D* — > D°7T with D° — > K + K~ or 7r + 7r _ , which is misidentified as D° — > Kit. These 
candidates are reconstructed at the right Q value for D* decays, but appear outside the D° 
mass region in Mk-k- Although it is hard to see these candidates in the lego plots of Figure [l], 
they show up as the enhancements in the projected Q distributions in Figure |5|. 

Figure |5| also shows the mis-reconstructed K + K~ and 7r + 7r~ mass, time and Q distribu- 
tions from our Monte Carlo studies as the cross-hatched histograms in the bottom six plots. 
The normalization is determined by our fitted values for A^k^-k from Equation |K]. Although 
the reflections are barely visible in the mx n and time distributions, they contribute a broad 
enhancement at 0.006 GeV in the Q distributions. Figure shows the fitted contribution of 
D° — > K + K~ and 7r + 7r~ misidentified decays scaled up by a factor of 20 and superimposed 
on the wrong-sign mass plot. The reflected signal is depleted in the D° mass signal region 
(indicated by arrows). More relevant to the mixing rate determination is Figure ^a which 
shows the combined Q distributions from D° — > K + n~ and D° — > K~tt + , but with tighter 
cuts around the D° mass. Clearly, very little of the excess remains in the central D° mass 
region. The fit attributes only about 34 candidates to the total wrong-sign signal. 

We have also investigated the effects of other charm backgrounds which might feed into 
our wrong-sign samples using Monte Carlo studies and re-plotting correctly identified states 
as if they were misidentified. The largest such source of background comes from doubly- 
misidentified decays of D° — > Kit or Ktttcti, in which the K and a it of opposite charge 
are both misidentified (as tt and K) by the Cerenkov detector. Although we explicitly cut 
against these misidentified decays in our data selection, a small fraction is expected to pass 
our cuts. Our Cerenkov measurements allow D° decays to be doubly misidentified around 
1.3% of the time. However, only about 15% of these candidates have an invariant mass 
within 20 MeV/c 2 of the D° mass. The selection cut on the reflected mass of each candidate 
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further reduces this background by about a factor of 20. Furthermore, since the background 
is very broadly distributed in mass, rather than peaked in the signal region, we expect the 
fit to respond only weakly by changing the wrong-sign ratios, probably at the level of a few 
times 10 -4 or lower. Since this background has an exponential decay time distribution, it 
will be interpreted as a signal for r dcs , and will not affect the measurement of r mix or r int at 
all. 

The remaining potential sources of charm background come from D* — > ttD decays with 
the D decaying to a mode other than Kir or Knirir. For the Kn mode, the D° — > K + K~ and 
7r + 7r~ were the most significant, and were handled as described previously. In addition, we 
have examined decays D° — > K~7t + 7t°, K~fi + u (doubly-misidentified), and D° — > 7r + 7r~7r° 
(singly-misidentified), which might contribute as background to D° — > Kn. As a general 
rule, the misidentification rates for these modes are similar to what was observed for the 
double misidentification above (all misidentification is dominated by the Cerenkov selection 
criteria for the kaon candidate), while the misidentified masses (in some cases after losing a 
neutral particle) are well outside the signal region. None of the other decays were seen to 
contribute a significant background to our data samples. 

In performing the fit, we discovered that the r^ cs and r m i X terms are strongly anticorre- 
lated with the ri nt terms, and strongly correlated with each other. Figure [8| demonstrates 
how these correlations come about in a hypothetical case where the interference contribution 
approximately cancels the contribution from pure mixing. This plot demonstrates that even 
when the full time evolution deviates only slightly from the pure exponential form of DCS 
decays, a large contribution from mixing can be present if it is offset by a destructive in- 
terference contribution. This implies that the fitted values for the interference contribution 
and the mixing contribution are strongly anticorrelated. 

Figure [5] illustrates the correlations in our particular fit by showing the likelihood con- 
tour plots for representative pairs of parameters for the D° — > K + n~ mode. These strong 
correlations account for much of the uncertainty in the wrong-sign ratios. Table [TV] gives the 
correlation coefficients for the different wrong-sign ratios. The correlations of these ratios 
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with all other parameters of the fit are negligible. We note that the correlations would be 
slightly reduced in an experiment with better efficiency at short decay times where there is 
good discrimination between r^cs and the other terms. 

VII. OTHER FITS 

Table || shows our primary results in the search for mixing. These results assume that CP 
violation can only occur in the interference terms of the fit, an assumption supported by most 
extensions to the Standard Model (see the discussion in Section ||). However, to answer any 
concerns about this assumption, we have also performed a fit in which the CP constraints 
are relaxed. Table |V] shows the results for the wrong-sign ratios of that 41 parameter 
fit. As expected, the central values for r dcs and r mix bracket the corresponding combined 
terms in Table 0, and all the fit errors have increased. Using the criterion Aln£ = 0.82 
(neglecting systematic errors), we calculate the one-sided, 90% C.L. upper limits to be 
rm ix (D° -> D°) < 0.74% and r mix (D° -> D°) < 1.45%. 

We note that the earlier measurement by the E691 collaboration assumed that the 
interference terms r int were negligible. Recently, there has been lively discussion concerning 
the validity of this assumption 0,§,^(J. Although some authors suggest that the phase 
between DCS and mixing amplitudes may be small, and therefore that the interference 
terms r int should also be small, we prefer to quote our results without this constraint. 
Nonetheless, to compare our measurements with the previous results from E691, we have 
performed a fit in which we set the interference terms to zero. Our results for mixing are 
r mix = 0.21±[J;^±0.02%, which is to be compared with the E691 result r mix = (0.05±0.20)%. 
The reduction of the fit errors from Table H is indicative of the strong correlations with the 
r int parameters, which are now fixed at zero. E691 also touched on this point by considering 
several different fixed values of the interference term (only one interference term was allowed 
in their model). Their results showed behavior similar to what we see in our data: strong 
correlation between wrong-sign ratios, and reduced fit errors when fixing the interference 



21 



term to zero. 

Finally, we explore the possibility that mixing is completely negligible, as one would 
expect from purely Standard Model contributions. In this case, we fit only for the DCS 
terms, obtaining r dcs {KTt) = (0.68±[];|f ± 0.07)% and r dcs (Km) = (0.25^34 ±0.03)%. The 
result for D° — > Kit demonstrates a two-sigma excess in the signal region which we believe 
is the result of real DCS decays. Figure [1(3 shows this excess after background subtraction. 
We note that our value for the DCS rate is consistent with the CLEO measurement || for 
the total wrong-sign rate: r ws (Kn) = (0.77 ± 0.25 ± 0.25)%. 



VIII. SYSTEMATIC UNCERTAINTIES 

Systematic uncertainties in the fit arise primarily from our modeling of the three functions 
e(t), R(Q), and Bj a i se {t). By using the right-sign data samples to estimate these functions, 
we have minimized our dependence on Monte Carlo models, but some uncertainty remains. 
The results of our studies of systematic uncertainties are summarized in Tables [VT] and |V11 . 



Below we describe the entries in Table [VI] which are the systematic errors obtained from 
studies on the fit of Table ||. The entries in Tables [VII| are obtained similarly. We find that 



the systematic uncertainties in the analysis are small compared with the statistical errors 
from the fit. 

The uncertainties in the first row arise from our estimates of the size of the reflection 
in Kir from misidentified D° — > K~K + and D° — > 7r _ 7r + events. Most of this uncertainty 
arises from our knowledge of the branching ratios for these modes. We have estimated this 
error as 20% of the difference between the results of fits with and without terms describing 
these reflections. 

The uncertainties listed in the third row ("statistics of effy & bkgd distrs") result from 
the uncertainties in our estimates of e(t), R(Q), and Bf a i se (t) due to the finite size of our 
right-sign data sample. The error bars on the corresponding histograms in Figures 0, |3|, 
and HI show the level of uncertainty involved. Statistical uncertainties on the model of the 
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function R(Q) (Figure |3[) have been greatly reduced by combining each D° candidate with 
pions from many different events, so that the uncertainties in this histogram are negligible in 
comparison with the uncertainties of Figures ^ and In order to propagate these statistical 
uncertainties to uncertainties on the fitted parameters, we perform many fits to the data, 
modifying each bin in each of the histograms of e(t), R(Q) and Bf a i se (t) by a Gaussian 
fluctuation with resolution given by the error bars. The rms spreads of the fitted parameters 
from 25 such fits are given in Table [VT]. 

The uncertainties listed in the fourth row of Table [VT] ( "binning of effy & bkgd distrs" ) 
are due to the fact that we have represented the e(t), R(Q), and Bf a i se (t) functions by 
binned histograms rather than smooth functions. We have, of course, tried to choose bin 
sizes small enough so that binning effects are not significant. In order to verify this claim, 
we replaced the histograms in the fit with smoothed functions which were derived from 
the histogram data, and repeated the fit. The differences between the parameter values 
with the smoothed functions and the parameter values with the histograms are quoted in 
Table |V| as the uncertainties due to binning. We expect this method to give an overestimate 
of the binning effect, since it also includes the effect of some statistical fluctuations in the 
measured histograms which are adjusted by the smoothing function. As anticipated, the 
binning effects are small. 

The uncertainties listed in the fifth row of the table ("time resolution") are due to the 
resolution on the measured decay time. Since the smearing is small, a good assumption is 
that e(t) is almost the same with and without smearing. For this analysis, the most likely 
resolution on the decay time is about 0.03 ps, with some measurements having a resolution 
as large as 0.08 ps. In order to quantify the error due to smearing, we replace the functions 
of Equation (|^) by functions convoluted with a fixed Gaussian resolution: 

B exp (U) = B° exp J dt e~(^W e - r * e(t), 

B mix (U) = B° mix J dt e-C^W t 2 e~ rt e(t), (17) 
B int {U) = Bl t J dt e -(*"*0W t e -r* e(t)) 
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where e(t) is obtained as in Section V and the integrals are obtained numerically. We then 
perform the fit with three different values for a : 0.02 ps, 0.05 ps and 0.08 ps. We quote the 
average of the central value differences (from the fits for 0.02 ps and 0.05 ps and from the 
fits for 0.08 ps and 0.05 ps) as variations in Table |VT], line 5. When the exponential lifetime 
is modified by our detector acceptance which is poor at low lifetimes, there is a "peak" at 
~ 0.5 ps. We observe that the time smearing affects the likelihood most near this "peak", 
while mixed events are most likely around higher values of decay time. Consequently, the 
DCS ratios exhibit the largest variation, while the mixing ratio is relatively stable. 

The uncertainties in the sixth line of the table ("mass resolution") come from the as- 
sumption of a constant Gaussian resolution in m(Kn) and m(K7nnr). In truth, the mass 
resolution should depend on the D° momentum and on the kinematics of the decay. We 
have studied the dependence on momentum and verified a noticable correlation between 
resolution and momentum. For the Kn decay mode, about 90% of the events have a mass 
resolution between 12 and 16 MeV/c 2 , with a tail reaching out to about 25 MeV/c 2 at high 
momentum. For the Ktitttt mode, the variation is much smaller, with all events exhibiting 
a mass resolution in the range 8.5 to 11 MeV/c 2 . To quantify this effect, we have varied the 
mass resolutions ±2 MeV/c 2 in the fit and recorded the maximum variations in wrong-sign 
ratios in Table [VT|. The fit results are quite insensitive to variation of the resolution on 
m(Kinnr), but change slightly with the resolution on m(K7r). A change in m(K7r) resolu- 
tion primarily affects rd cs (Kir), but because of the correlations in the fit, it will also alter 
r mix and r dcs (Kimii), as shown. 

Some other assumptions and biases in our fit model bear further comment. First of all, we 
have assumed that the efficiency function e(t) for reconstructing a D° from a D* decay (signal 
terms from Eq. (Q)) is the same as the efficiency function for reconstructing a primary D° 
(random pion term from Eq. (|11|)). Since our reconstruction and selection criteria are only 
weakly dependent on the D° production kinematics, we expect to find very little difference 
in efficiency for these two sources of D mesons. Studies of reconstructed D° decays which 
are not associated with a bachelor pion appear to confirm that the difference is negligible. 
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Secondly, the Kirinr final state may result from different resonant substructures in the 
Cabibbo-favored and DCS amplitudes. This can, in principle, lead to different efficiencies 
for the DCS and interference terms in the fit. (The two-body Ktt mode is, of course, 
immune to this problem.) Once again however, the fact that our reconstruction depends 
only slightly on the decay kinematics leads to effects at the level of only 1%. The very 
similar time dependence of the efficiency functions for the Ktt and Ktttttt final states (Fig. ^) 
demonstrates how little e(t) depends on the D decay. 

We are also aware that training the D* neural net on a sample of right-sign D* decays 
can, in principle, produce a small bias in that sample (but not the wrong-sign samples 
which were not used for training the net). Careful selection of input variables for the neural 
net that are not correlated with Q (our variables depend only on parameters that describe 
the bachelor pion) should prevent any significant bias. We have investigated this effect by 
subdividing the training sample into 10 subsamples, training a neural net on one subsample, 
and applying the resulting net to the remaining samples as a test of the bias. We then 
repeat the process on each subsample to get a better statistical average. By comparing 
the sensitivity (measured as S/\fB) of the training samples with the sensitivity of the test 
samples, we determine the level of bias. We find that the number of right-sign signal events 
could be biased upwards by about 1%. This is a negligible effect compared to our statistical 
error. 



The last line in Table VI shows the contribution from all the systematic errors added in 



quadrature. These totals are less than half the size of the statistical errors in Table ||. 



IX. DISCUSSION 



At the current level of sensitivity, mixing searches begin to constrain some models [|2T 
There are also other search methods that are promising. Using the same D* decay chain 
to identify the produced D meson, but looking at semileptonic decays of the D, is one 
possibility. Although semileptonic decays are harder to reconstruct due to missing neutrinos, 



25 



they are not subject to contributions from DCS amplitudes, and therefore do not suffer from 
the main limitations discussed in this paper. In a separate publication we describe such a 
search || with the result r m i X < 0.50% at the 90% CL. The possibility exists for even higher 
statistics searches in future experiments. Alternatively, it may be possible to detect mixing 
via the lifetime difference between the two physical eigenstates by comparing the measured 



lifetimes for different CP final states ||22|| . Of course, this approach will only detect mixing if 
it is associated with a substantial lifetime difference as opposed to mixing that only results 
from a mass difference. We are investigating this method as well. Finally, the cleanest signal 
for mixing might be found at a r-charm factory which produces D° — D pairs on resonance. 
As has been discussed previously ||23|| , certain hadronic final states from these D° — D° pairs 
can only be produced by mixing and not by DCS amplitudes. We remain hopeful that one 
of these techniques may be used to detect D° — D mixing, and thus provide information 
about the existence of new physics. 



X. SUMMARY 

We have searched for evidence of D° — D mixing and DCS decays by looking for wrong- 
sign decays in the decay chain D* — > 7rD with D — > Kir or D — > Ktttttt. Our results are 
summarized in Table [VIII| . 

We have seen no evidence for mixing in either D° decay mode. The results of a maximum 
likelihood fit to the data are given in Table ||. The possibility of additional sources of wrong- 
sign decays from DCS amplitudes limits our sensitivity for detecting mixing alone. Using the 
criterion Aln£ = 0.82, we calculate the one-sided, 90% C.L. upper limit to be r mix < 0.85%. 
If, in order to account for the most general case possible, we relax the assumption that CP 
is conserved in the mixing and DCS terms of the fit (as in Table |V[), we calculate the upper 
limits for mixing to be r mix (D° -> D°) < 0.74% and r mix (D° -> D°) < 1.45%. 

Our quoted sensitivity to mixing is similar to that of Fermilab E691, but our analysis is 
notably more general in its assumptions concerning DCS-mixing interference and CP viola- 
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tion. Assuming no DCS-mixing interference constrains the mixing and DCS contributions 
much more severely, but we do not feel this assumption is justifiable. Nevertheless, for com- 
parison we include the mixing results for this case in Table |V111| . All our results for the Kn 
final state are also consistent with the CLEO measurement of r ws (Kir) = (0.77±0.25±0.25)% 
H for wrong-sign decays. In particular, if the mixing amplitude is set to zero, we find a 
two-sigma enhancement in the Kir mode and no significant enhancement in the K r n r n r n mode: 
r dcs {Kix) = (0.68l£! ± 0.07)% and r dcs {K m ) = (0.25i£| ± 0.03)%. 
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TABLES 

TABLE I. Correction factors for the difference in integrated reconstruction efficiencies associ- 



text. 



Cmix(KTr) 0.388 

Cint(KTT) 0.499 

c m ix(Kirinr) 0.359 

c int (KTnnr) 0.473 
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TABLE II. Signal and background parameters for the fit described in Section [V|, which as- 
sumes no CP violation in either the DCS or mixing terms. Thus, we have used the constraints 
r dcs (D° -» K+TT-) = r dcs (D° K~n+), r dcs (D° -> K+tttttt) = r dcs {D° K-vrvrvr), and 
W(X>° - £°) = r mix (D° £>°). 



7T 



TTTTTT 



K+TTTTTT 



A 

A 



2269 



+49 
-49 



2966 



+56 
-56 



746 



338 



+33 
-33 

+24 

-24 



423 



+27 
-27 



131411 

31ll£ 

278lli 



1677 



+42 
-42 



368 



356 



+23 
-23 

+21 
-21 



K~7T- 



D c 



K + TT- D ® — ► K~TTTT7r 



K + TTTTTT 



r dcs (%) 


Q0+ 1 - 20 
u - yu -1.09 


±0.44 


-n 20+ 1 - 17 

u - zu -1.06 


±0.35 


frnix (%) 




0.39l°;l 


±0.16 




r in t(%) 


— 0.46±g;|| ± 0.41 


-0.841?;^ ± 0.43 


-0.291q:96 ± °- 37 


— 0.25±g;|I ± 0.36 


A v 


707+32 

' '-32 


74Q+ 30 
' 4y -30 


3231^ 


315±H 


A 


2431^ 


QQO+22 

-J<JO_22 




2381$ 




601^ 


49l{^ 









K--K+ 




K TTTTTC 


K + TTTTTT 


Ai (c 2 /GeV) 


_o nq+0.81 


70+O.68 
°-'°-0.68 


-2-861^ 


-2 65+ - 70 
z - oo -0.70 
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TABLE III. Mass parameters for the fit described in Section |V], which assumes no 
CP violation in either the DCS or mixing terms. Thus, we have used the constraints 
r dcs (D° -» K+ir~) = r dcs (D° K~n+), r dcs (D° -► K+tttttt) = r dcs {D° -► ^"7^), and 
r mix{D® — ► -D ^) = r m ix(D® — > D®). All values are in MeV/c 2 . There are systematic uncer- 
tainties on these parameters that are bigger than the statistical errors shown here, but they have 
inconsequential effects on the parameters of Table |lT[ 



m D o 1865.8±g;^ 
(?k-k 15.16_ 016 

&K7V7T7V 

10.76I ; 15 

Qd* 5.921q;q]; 

n 7fi+o.oi 

°Q °- 7 °-o.oi 



TABLE IV. Correlation coefficients for the wrong-sign ratios from the fit of Table |TJ. Only the 
lower halves of the symmetric matrices are shown. Correlations with the fit parameters that are 
not shown are negligible. 

r dcs (Kn) r dcs (K-Kmr) r int {K--K + ) r int (K + ir-) r int (K~innr) r int (K + mnr) r mix 



r dcs (Kir) 


1.00 










r dcs (Kirir-K) 


0.58 


1.00 








r int (K~ir + ) 


-0.92 


-0.68 


1.00 






r int (K + TT-) 


-0.90 


-0.68 


0.95 


1.00 




r int {K~ 7T7r7r) 


-0.71 


-0.90 


0.84 


0.85 


1.00 


r int (K + innr) 


-0.70 


-0.90 


0.82 


0.83 


0.95 




0.78 


0.74 


-0.92 


-0.93 


-0.92 
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TABLE V. Fit results for the wrong-sign ratios of the most general fit, with no assumptions 
about CP. 



D -» K--K+ D -» K--KiriT D° -» L>° -» K+vrvrvr 

r dcs (%) 0.80j£$ ± 0.47 -0.67^ ± 0.41 1.26^ ± 0.49 0.33^$, ± 0.32 

rw(%) 0.18l°;| ± 0.17 0.70±g:f| ± 0.18 

r in t{%) -0.1lii:?6 ±0.43 0.22±i;i| ± 0.41 -1.46^ ±0.47 -0.89±J$j ± 0.35 



TABLE VI. Systematic uncertainties for the key parameters in the fit of Table ||. This fit 
describes the case where we have allowed for CP violation only in the interference term. Entries 
are explained in the text. 





r dcs (K7r) (%) 


r dcs (K7T7nr) (%) 


Tmix (%) 


fit value 


o qn+ L20 

u - yu -1.09 


-0.20±1& 


0.39l°;l 


K + K~, tt + tt~ reflections 


±0.02 


±0.02 


±0.01 


statistics of effy & bkgd distrs 


±0.27 


±0.17 


±0.09 


binning of effy & bkgd distrs 


±0.25 


±0.21 


±0.11 


time resolution 


±0.17 


±0.19 


±0.06 


mass resolution 


±0.18 


±0.11 


±0.05 


total systematic uncertainty 


±0.44 


±0.35 


±0.16 
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TABLE VII. Systematic uncertainties in the no-mixing fit, corresponding to the Standard 
Model case. Entries are explained in the text. 





r dcs (Kir) (%) 


r dcs (KinTir) (%) 


fit value 




n oc+0.36 
U - ZO -0.34 


K + K~, 7r + 7r~ reflections 


±0.04 


±0.00 


statistics of effy & bkgd distrs 


±0.02 


±0.02 


binning of effy & bkgd distrs 


±0.01 


±0.00 


time resolution 


±0.01 


±0.01 


mass resolution 


±0.04 


±0.02 


total systematic uncertainty 


±0.07 


±0.03 



35 



TABLE VIII. A summary of values from our four fits. The top line describes the most likely case 
for extensions to the Standard Model which produce large mixing. The second line describes our 
most general fit, providing the most conservative results. The third line matches the assumptions of 
previous experiments, which we do not feel are justifiable. The bottom line describes the Standard 
Model case. Results from other experiments are also listed for comparison. 



Fit type 




This Result 


Other Comparable Result 


CP violation only in 
interference term 




= (0-39i£| ± 0.16)% 


rmix = (0.1118:1?)% (E791 §) 
(Semileptonic decays) 


Most general, 
no CP assumptions 


r ■ (D° - 

1 mix \ J -' 
^mix^fl 


+ D°) = (0.18±g;|| ± 0.17)% 
>D°) = (0.70l°;l ± 0.18)% 




No CP violation, 
no interference 


Vmix 


= (0.21±g;gg ± 0.02)% 


r mi x = (0.05 ±0.20)% (E691 §) 


No mixing 


rdcs{K'. 


w) = (0.68±8;i| ± 0.07)96 


r ws (Kir) = (0.77 ± 0.25 ± 0.25)% 



TdciKirinc) = (0.25±g;l| ± 0.03)% (CLEO g) 
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FIGURES 




FIG. 1. Plots of Q (defined in the text) versus the candidate D mass for right-sign D — > Kir 
(top-left), right-sign D — > Kiritir (top-right), wrong-sign D — > K7r (bottom-left), and wrong-sign 
D — > Kit-kit (bottom-right). Clean signals are apparent in both right-sign plots. In all four plots, 
the bands of events at m(Kir), m(KiririT) w 1.87 GeV/c 2 are due to real D decays combining with 
random pions to give false D* candidates. 
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FIG. 2. Reconstruction efficiencies for D° — ► i^7r (left) and Z) — > K-kittt (right), as measured 
from the right-sign data samples. The low efficiency at short decay times is typical of fixed-target 
experiments which identify charm decays by a secondary decay vertex. The drop in efficiency at 
long decay times is due to our selection criteria which remove decays occuring in downstream target 
foils. Efficiencies for charge conjugate final states (e.g., e K - w + and e K + n -) are observed to be the 
same within errors, and have been combined in the above plots. The vertical scales are arbitrary. 
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FIG. 3. Histograms show distributions of Q = m(Kirir) — m(Kir) — m(7r) (top) and 
Q = m(Kiririnr) — m(KiririT) — m(w) (bottom) for wrong-sign D candidates in the mass range 
1.835 to 1.895 GeV/c 2 . The points with error bars show the distributions from combining D 
candidates and 7r's from separate events, normalized to the histograms. These distributions are 
used to represent R(Q) in the likelihood fits. 
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FIG. 4. The measured decay time distribution for D — > i^7r candidates (left) and D — > Ktttttt 
candidates (right), taken from the sidebands 1.77 < m/j < 1.81 GeV/c 2 and 1.93 < m/j < 1.97 
GeV/c 2 . These distributions are used to represent Bf a i se (t) in the likelihood fit. 



40 



00 

a: 400 




1.8 1.9 

m K „ (GeV/c 2 ) 





2 4 

Decay time (ps) 




2 4 

Decay time (ps) 




2 4 

Decay time (ps) 




2 4 

Decay time (ps) 



1500 F~ 
1000 
500 





0.01 0.02 

Q (GeV) 



0.01 0.02 

Q (GeV) 




0.01 0.02 

Q (GeV) 




0.01 0.02 

Q (GeV) 



FIG. 5. Projections of the four D — > Kir data samples onto each of three distributions mx n , t 
and Q. Data are from the D° mass range (1.770 < m(Kir) < 1.970) GeV/c 2 . Solid curves show the 
projections of our primary fit, summarized in Table ||. The broad component of the peaks in the WS 
Q plots is described well by reflections of K + K~ and ir + ir~ signals. MC simulations of reflected 
K + K~ and 7r + 7r~ signals normalized to the fit values of Akk,-k-k are shown as cross-hatched 
histograms in the wrong-sign plots. 
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FIG. 6. Projections of the four D —* Kmrn data samples onto each of three distributions 
"i_ft:7r7r7r ; and Q. Data are from the D° mass range (1.770 < m(K-KinT) < 1.970) GeV/c 2 . Solid 
curves show the projections of our primary fit, summarized in Table |J. 
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FIG. 7. The sum of the wrong-sign mass distributions in the lower two rows of Figure ||. The 
cross-hatched histogram is the reflected mass distribution of D° -> K + K~ and D° — > 7r + 7r decays 
from Monte Carlo normalized to 20 times the amount favored by our primary fit, summarized in 
Table |5. Notice the depletion of reflected signal in the D° mass signal region. Figure ||a shows the 
Q distribution for events within the restricted mass region indicated by the arrows. 



43 



200 nv 



CD 

a 
c 
00 



en 

c 

o 



50 



00 



50 








0.5 1 1.5 2 2.5 
decay time (psec) 

FIG. 8. A hypothetical plot of the time dependence of wrong-sign decays taken from [[?]]. The 
dashed line represents the DCS contribution. The dotted line shows the contribution due to 
mixing. The dash-dot line shows the contribution from destructive interference of DCS and mixing 
amplitudes when the interference is 30% of its maximum. The solid line is the sum of all three 
contributions. The vertical scale is arbitrary. 
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FIG. 9. Likelihood contours corresponding to Aln£ = 0.5 and 2.0 for the fit of Table ||, 
illustrating the correlations among the three parameters r^^Kir), r m i x and ri n t(K + ir~). Strong 
correlations among these parameters are apparent. The correlations among other wrong-sign ratios 
are similar. 
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FIG. 10. Distribution in Q for wrong-sign D — > Kit candidates in the mass range 1.845 to 1.885 
GeV/c 2 . The plot has been background subtracted using the fit with no mixing (Standard Model 
case). The Gaussian overlay shows the size of the signal attributed to DCS decays by the fit. 
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